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ABSTRACT 


This  final  report  on  the  Grant  No.  F49620-97- 1-0148  consists  of  somewhat  abridged 
versions  of  all  previously  submitted  progress  reports,  placed  in  individual  sections  o  - 
lowed  by  a  section  on  the  progress  made  in  the  last  12-month  performance  period  and  on 
promising  future  directions  for  our  work. 


A  broad  understanding  of  all  aspects  of  image  formation,  detection,  and  processing 
in  information  theoretic  terms  has  been  achieved  in  our  studies  under  direct  support  ot 
the  grant.  Specifically,  we  have  (i)  shown  how  prior  knowledge  about  the  object  being 
imaged,  particularly  about  the  support  of  its  intensity  distribution,  can  be  regarded  as  a 
gain  of  information;  (ii)  computed  the  theoretical  limit  on  the  information  throughpu  , 
i.e.,  the  channel  capacity,  of  an  astronomical  imaging  system;  (iii)  computed  th^retical  y 
and  in  simulation  the  detailed  characteristics  of  the  dynamics  of  noise  versus  information 
in  imaging  under  turbulence  when  support  information  is  employed  as  a  post-processing 
tool;  and  (iv)  gained  a  preliminary  understanding  of  the  inverse  relation  between  noise  and 

information. 
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I.  REPORT  ON  PROGRESS  MADE  DURING  THE  PERIOD  3/97-8/97 


The  first  recognition  that  image  formation  may  be  usefully  described  from  the  view¬ 
point  of  Shannon’s  statistical  information  theory  dates  back  to  Fellgett  and  Linfoot’s  pio^ 
neering  work  [1]  in  1955.  They  argued  for  two  competing  viewpoints  on  the  assessment  of 
the  quality  of  image  formation; 

1.  How  similar  is  the  image  to  the  object?  This  could  be  studied  by  defining  a  metric, 
that  measures  the  departmre  of  the  image  from  the  object,  in  terms  of  optical  system 
transmission  factors. 

2.  How  faithfully  does  the  system  image  a  class  of  objects,  in  the  sense  of  the  closeness 
of  the  information  content  of  the  image  set  relative  to  that  of  the  object  set? 

By  shifting  attention  from  a  single  object  to  a  whole  class  of  objects,  the  second 
viewpoint  provides  for  a  very  powerful  characterization  of  the  imaging  system  as  a  conduit 
for  the  passage  of  information.  This  is  an  inherently  more  useful  viewpoint  if  the  objective 
of  study  is  the  optimality  of  the  design  of  an  optical  system  to  image  a  variety  of  objects. 
For  example,  if  a  whole  class  of  satellites  which  all  contain  the  same  gross  features  is  the  set 
of  objects  for  which  an  imaging  system  is  being  designed,  then  it  is  clear  that  the  second 
viewpoint  provides  a  statistical  criterion  for  the  best  design.  Indeed,  since  one  does  not 
typicaUy  know  in  advance  the  details  of  the  object  being  imaged,  the  question  of  how  close 
its  image  is  to  itself  cannot  even  be  answered. 

It  is  this  motivation  that  characterizes  our  work,  based  on  the  use  of  statistical  infor¬ 
mation  entropy  and  other  related  ideas. 


A.  Object  Set 


We  can  specify  the  object  set  statistically  by  a  probability  distribution  P[d{f)]  on 
the  object  spatial  spectrum  0(/),  the  latter  defined  as  the  Fourier  transform  of  the  two- 
dimensional  intensity  distribution  of  the  object.  The  object  illumination  is  assumed  inco- 
herent  here,  as  is  true  for  most  astronomical  sources  or  solar-illuminated  satellites.  The 
object  spatial  spectrum  at  a  spatial  frequency  is  directly  proportional  to  the  complex  fringe 
visibihty  for  interference  of  light  received  at  two  apertures  with  vector  separation  propor¬ 
tional  to  the  spatial  frequency  vector.  This  is  the  essence  of  the  Van  Cittert  -  Zernike 
theorem,  which  governs  interferometric  image  reconstruction. 


We  begin  with  a  circular  Gaussian  model  for  P[0(/)]  : 


pio(/)i=n 


— e 


-|d(/)-<6(/)>|V(2<^o(/)) 


Y27r<T^(/) 

where  the  product  is  over  a  discrete  set  of  spatial  frequencies  and  <To(/)represents  the 
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standard  deviation  of  the  object  spectrum  at  frequency  /.  The  question  we  asked  and 
successfully  answered  is  the  following:  Relative  to  this  object  set,  what  is  the  statistical 
information  successfully  transmitted  by  a  simple  optical  system  working  under  conditions  ot 
turbulence  induced  phase  aberrations  at  high  light  levels?  In  order  to  answer  the  question, 
we  must  first  discuss  how  an  optical  system  processes  an  image  under  turbulent  conditions, 
but  without  the  complications  of  either  the  detector  or  low-light-level  shot  noise. 


B.  Ifnaging  under  Turbulent  Conditions 


An  incoherent  imaging  system  is  a  linear  system  in  the  sense  that  the  image  of  a 
collection  of  incoherent  point  sources  is  a  superposition  of  the  elementary  imag^  (point 
spread  functions  (PSFs))  of  the  individual  point  sources.  Since  an  incoherent  object  may 
be  regarded  as  a  contiguous  collection  of  such  point  sources,  the  image  and  object  intensity 
distributions  may  shown  to  be  related  through  the  PSF  by  a  convolution.  In  the  ^uner 
space,  therefore,  the  object  and  image  spatial-spectrum  amplitudes,  /(/)  and  0(/j, 
related  by  a  simple  product  involving_^the  Fourier  transform  of  the  PSF,  also  known  as  the 

optical  transfer  function  (OTF),  'H{f): 

i{f)  =  oifMf).  (2) 

The  OTF  of  an  optical  system  under  turbulent  conditions  is  on  average  a  product  of  the 
system  OTF  in  the  absence  of  turbulence  and  the  OTF  that  can  be  directly  associated 
with  the  turbulent-induced  amplitude  and  phase  fluctuations. 

Of  course,  under  turbulence,  there  arise  fluctuations  in  the  recorded  image  intensity 
spectrum,  since  the  OTF  suffers  from  fluctuations.  We  may  see  from  Eq.  (2)  that  the  image 
noise  is  multipUcative,  which  is  a  major  analytical  complication.  We  adopt  an  approximate 
procedure  to  include  this  noise  in  the  analytical  treatment  that  follows. 

As  we  have  indicated,  the  mean  OTF  is  a  product  of  the  system  OTF  no{f)  the 
mean  atmospheric  OTF  {'Hatmif))' 

{'H{f))  =  'Ho{f){'Hatm{f))' 

The  sum  of  the  variances  of  the  real  and  imaginary  parts  of  the  OTF,  which  we  simply  call 
the  psuedovariance  of  the  OTF,  may  be  approximately  computed  with  a  slightly  modifaed 
structure  function  for  the  turbulence-induced  random  phase  distribution  m  the  pupil.  Ihe 

''  var['H(/)]  «  [1  -  exp{-C\'^F^\f\Vrl)]\no{f)\\  (4) 

where  F  is  the  system  focal  length,  A  the  light  wavelength,  ro  the  atmospheric  coherence 
diameter,  and  (7  is  a  constant  of  order  3.44.  The  correlation  of  the  fluctuations  of  the  OTF 
at  two  different  spatial  frequencies  may  also  be  computed  analytically  with  the  resu  , 

{Anif)A'H*  (f ))  «  ysx[H{f)]Sfj, ,  (5) 
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V  . 


under  the  assumption  of  strong  turbulence  tq  — >•  0.  In  other  words,  in  this  limit,  two 
different  spatial  frequencies  have  uncorrelated  noise. 

It  is  worth  noting  that  the  preceding  analysis  is  a  conditional  analysis  of  the  mean  and 
the  fluctuations  in  the  image,  conditioned  on  the  precise  specification  of  the  object.  We 
must  bear  in  mind  that  an  object  is  drawn  from  a  statistical  set,  the  so-called  object  set. 
But  it  is  the  conditional  statistical  distribution  of  this  sort  that  governs  the  degradation 
of  information  in  an  object  by  the  random  fluctuations  of  the  processing  system.  When 
the  related  conditional  information  is  averaged  over  the  object  set,  we  get  a  measure 
of  what  portion  of  the  information  in  the  image  is  vmreliable  and  therefore  not  faithfully 
transmitted  from  the  object  set.  Shannon  defined  the  successfully  transmitted  information 
as  the  following  mutual  information: 

H[0-,  I]  =  H[I]  -  H[l/0]  =  H[0]  -  H[0/I],  (6) 

where  the  object  information  content  is  given  by 

H[0]  =  -^P[d]logP[dl  (7) 
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with  the  object  probability  distribution  given  by  expression  (1).  Similar  expression  may 
be  given  for  the  image  information  H[I].  For  the  conditional  information  H[I/0]  (or 
H[0/I]),  one  has  a  similar  expression: 

H[I/0]  =  -  (8) 

6,i 

where  P[//0]  is  the  conditional  probability  distribution  of  the  image  spectrum,  given  a 
specified  object  spectrum.  If  we  make  the  tenable  asstunption  that  the  OTF  %(/)  in 
Eq.  (2)  is  a  result  of  a  linear  superposition  of  many  independent  contributions,  as  true  in 
the  strong-turbulence  limit,  then  the  conditional  statistics  of  the  image  spectrmn  can  be 
accurately  described  by  a  circular  Gaussian  probability  distribution  with  the  mean  and 
the  variance  given  by  Eqs.  (3)  and  (4). 

With  the  preceding  justifications  and  approximations,  we  can  now  discuss  two  cases 
of  some  practical  interest. 


1.  Low  Variability  in  the  Object  Set,  <to(/)  0 


In  this  case,  the  image  spectrum  probability  distribution,  which  requires  ^integration 
of  the  object  probability  distribution  with  the  conditional  distribution  P[I/ O]  over  the 
object  reahzations,  reduces  approximately  to  a  product  of  two  Gaussian  distributions. 
Such  integrations  are  straightforward.  The  final  result  for  the  successfully  transmitted 
information,  the  mutual  information  of  Eq.  (6),  is  the  following: 
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(9) 


H[0-, I]  ='£ioi  [1  + 

f 

where  the  conditional  variance  in  l^he  image  when  the  object  is  known  a  priori 

takes  the  form 

<'l/o“K'5(/))Pl«o(/)p.  (10) 

There  are  three  remarks  in  order  here.  Note  that  frequencies  beyond  the  system 
cutoff  are  excluded  from  Eq.  (9),  since  for  such  frequencies  no  information  is  transmitted 
by  the  system.  When  the  conditional  variance  in  the  image  <t//o  vanishes,  the  system 
transmitts  perfectly  the  full,  infinite  amount  of  information  there  is  in  each  of  the  infinitely 
accurately  specified  objects.  (This  can  be  made  finite  by  requiring  only  a  certain  finite 
decimal  accuracy  of  the  object  spectrum  specification.)  Finally,  when  the  object  variability 
goes  away,  <to(/)  0,  then  there  is  no  information  to  transmit.  A  fixed  object  carries  no 

information  in  the  sense  of  its  potential  to  carry  a  signal. 


2.  Infinitely  Strong  Turbulence,  ro  0 


In  this  hmit,  the  information  successfully  transmitted  by  the  optical  system  becomes 
independent  of  the  object  set  information,  in  particular  the  variance  of  the  object  set.  This 
is  understandable,  since  in  the  strong-turbulence  limit  the  atmosphere  filters  out  all  but 
the  lowest  spatial  frequencies,  and  so  it  is  expected  that  object  variability  will  have  little 
bearing  on  the  information  actually  passed  on  to  the  image. 

II.  REPORT  ON  PROGRESS  MADE  DURING  THE  PERIOD  9/97-8/98 

The  work  during  this  period  was  focused  on  gaining  the  vital  understanding  of  how 
an  a  priori  constraint  like  the  knowledge  of  the  support  of  the  image,  to  be  simply  called 
the  support  constraint  hereonwards,  can  lead  to  improved  statistical  information  in  the 
image.  An  indirect,  although  somewhat  more  traditional,  approach  that  we  first  took 
was  to  evaluate  the  reduction  of  noise  brought  about  by  the  imposition  of  the  support 
constraint.  We  summarize  below  only  the  salient  features  of  this  work,  but  leave  its  more 
complete  discussion  to  the  refereed  publication  [3]  accompanying  this  Report.  The  second 
part  of  the  report  is  a  discussion  of  our  more  direct  approach  to  the  problem,  in  which  we 
discuss  the  information  content  of  the  image  before  and  after  the  imposition  of  the  support 
constraint. 
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A.  Noise  Reduction  in  Image  Deconvolution  with  Support  Constraint 


.  ^ 


Use  of  a  priori  constraints  is  often  made  in  image  reconstruction  algorithms  to  improve 
the  final  image  beyond  what  the  limited  observational  data  can  generate.  One  way  in  which 
the  improvement  can  take  place  is  through  noise  reduction  that  is  directly  enabled  by  the 
a  priori  information.  We  summarize  here  how  the  knowledge  of  the  support  of  the  object 
can  lead  to  significant  noise  reduction  in  a  linear  deconvolution  of  raw  images  that  are 
degraded  by  atmospheric  turbulence. 

The  spatial  spectra  of  the  object  and  the  image,  0{u)  and  I{u)  are  related  very  simply 
through  the  optical  transfer  function  (OTF)  if(it), 

I{u)  =  0{i[)H{u).  (1) 

For  well-corrected  optics  and  at  high  light  levels  with  noiseless  detectors,  the  image  noise 
in  the  spatial  frequency  (u)  plane  is  governed  by  the  fiuctuations  of  H{u)  arising  from 
turbulence. 

When  the  image  is  constrained  to  have  a  known  finite  support  described  by  the  sup¬ 
port  function  s(x),  equal  to  1  for  x  inside  the  support  and  0  outside,  then  the  support- 
constrained  and  unconstrained  images  are  related  by  the  multiplicative  factor  ^(a:)  in  the 
physical  domain  and  by  convolution  with  the  support  spectrum  S”(m)  in  the  Fourier  plane: 

h{u)  =  j lid-  u')Siu')cPu'.  (2) 

The  spatial  spectrum  (simply  called  spectnun  hereafter)  of  the  image  noise  can  be 

constructed  in  terms  of  the  two  mean  squares  N{u)  =  (|/(«)  -  (/(u))P)  and  M{u)  = 
([/(u)  — (/(u))]^).  The  relation  between  the  noise  power  spectra  before  and  after  imposition 
of  the  support  condition  follows  from  Eq.  (2)  supplemented  by  the  result  (1).  The  following 
approximate  convolution  relation  may  be  shown  [3]  to  hold  for  the  mean  squared  modulus: 

Ns{d)  =  j  dPuxNid  -  (3) 

where  represents  the  phase  structure  function  for  atmospheric  turbulence. 

A  powerful,  although  somewhat  approximate,  picture  of  the  relation  (3)  between  the 
constrained  and  unconstrained  noise  spectra  emerges  when  one  expands  iV(u  —  «i)  in  a 
Taylor  series  around  ui  =  0.  Such  an  expansion  leads  to  an  expansion  of  Eq.  (3)  in  deriva¬ 
tives  of  iV(u)  with  increasing  derivative  order  from  one  term  to  the  next.  The  coefficients 
of  the  derivative  terms  are  expressed  as  tensor  moments  of  the  support  spectrum.  With 
the  retention  of  only  the  first  three  terms  in  the  expansion,  we  obtain 

AN{u)  =  N,{u)  -  iV(u) 

=  B  •  ViV(u)  +  :  VViV(u),  (4) 
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where 


is  a  vector  and 


J?  =  - 

j  <fu 

(5) 

(6) 

is  a  second-rank  tensor. 


Note  that  the  equation  (4)  formally  describes  a  noise  transformation  that  contains  a 
drift,  or  ballistic  transport,  term  and  a  diffusive  spreading  term.  The  physical  implications 
of  this  viewpoint  are  noted  in  detail  in  Ref.  3. 

For  an  inversion-symmetric  support  like  a  circle  (or  an  ellipse),  the  drift  vector  B 
vanishes  by  symmetry.  The  noise  transport  induced  by  the  support  constraint  in  the 
spatial  frequency  plane  is  then  purely  diffusive.  Such  transport  enables  a  reduction  of 
noise.  All  one  has  to  do  is  to  leave  unchanged  the  original,  unconstrained  data  at  those 
spatial  frequency  locations  where  the  transformed  data  are  noisier  than  the  original  data, 
as  pointed  out  by  Tyler  and  Matson  [4]  in  their  simulations  of  the  same  problem.  With 
this  picture,  we  can  easily  evaluate  the  total  noise  reduction  possible  in  one  application  of 
the  support  constraint.  It  turns  out,  as  shown  in  detail  in  the  attached  manuscript,  that 
we  have  an  analytical  expression  for  the  ratio  of  the  total  noise  reduction  to  the  total  noise 
in  the  original  image: 


"wW" 


TTcr 

MM 


e  2C(A//ro)2 


(7) 


where  a  is  the  diameter  of  the  support,  Xf/ro  is  the  diameter  of  the  atmospheric  seeing 
disk,  and  (7  is  a  number  of  order  10. 

Note  from  Eq.  (7)  that  the  best  fractional  noise  reduction  is  obtained  when 
the  support  is  comparable  in  size  to  the  seeing  disk.  Equation  (7)  represents  one  of  the 
most  important  results  of  the  work,  and  is  confirmed,  more  or  less  quantitatively,  by  the 
simulations  of  Tyler  and  Matson  [4].  In  the  next  subsection,  we  begin  a  description  of 
this  problem  from  the  viewpoint  of  information  theory.  Details  may  be  found  in  another 
accompanying  article  [5]. 


B.  InforTnation  DynctTnics  in  Sceing-Limited  Image  Deconvolution  with  Support  Constraint 

Consider  the  mutual  information,  as  defined  by  Shannon  to  be  the  information  suc¬ 
cessfully  transmitted  through  a  communication  channel.  The  mutual  information  T(u)  at 
spatial  frequency  u  of  the  image  spectrum  is  the  difference, 

I{u)  =  Hoiu)  —  7f<,|i(u),  (8) 
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of  the  (unconditional)  object  information  Uo{u)  and  the  conditional  object  info^ation 
UnXiiu),  conditioned  on  the  observation  of  the  image  spectrum.  According  to  Shannon, 
each  kind  of  information  is  given  by  averaging  the  logarithm  of  the  corresponding  proba¬ 
bility  density  function  (pdf): 

Uo{u)  =  -{\ogPo{0{u)))  =  -  j  Po{0{u))logPo{0{u))<PO{u); 

^  .(^)  =  -{log Po\i{0{u)\I{u)))  =  -  f  P{Oiu),I{u))lo&Po\i{0{mi^))^0{u)d^I{u). 

(9) 

The  actual  values  that  the  statistical  variables  o,  i  can  assume  are  always  denoted  by  the 
corresponding  upper-case  symbols  O,  I. 

The  pdf’s  are  related  because  of  the  imaging  equation  (1).  For  a  turbulent  atmosphere, 
as  true  for  a  groimd-based  imaging  system,  the  OTF  H  has  random  variations  that  can 
be  described  adequately  by  the  Kolmogorolf  hypothesis  [6]. 

An  application  of  the  support  constraint  in  the  image  domain  changes  the  image 
spectrum  by  means  of  a  convolution  of  the  original  image  spectrum  and  the  spectrum 
of  the  support  function.  Since  the  object  spectrum  is  unchanged,  the  apphcation  ot  the 
support  constraint  causes  a  change  in  the  system’s  mutual  information  (8)  by  amount 

AIM  =  - [will 

It  is  enough  for  our  purposes  thus  to  compute  the  change  in  the  negative  conditional 
information  —'Ho\i- 

Assuming  that  the  change  of  pdf’s  under  the  support  constraint  is  small,  we  can  c^ry 
out  a  first-order  analysis  for  the  change  AI.  After  some  tedious  algebra,  one  may  show 
that  correct  to  the  lowest-order  changes  of  the  pdf’s,  the  change  in  information  throughput 
is  given  by  the  simple  relation 


AI(u)  = 


[APi|„(/(«)10(u))]  Po(0(u)) log  Poii{0{ii)\I{u))<fO{u)<fl{u). 


(11) 


The  conditional  pdf  Piu,  when  the  support  constraint  is  not  applied  yet,  is  easily 
evaluated  in  terms  of  the  known  object  pdf  and  the  pdf  of  the  OTF  which  we  can  compute 
based  on  the  Kolmogoroff  hypothesis  of  turbulence.  Indeed  from  Eq.  (1),  since  for  a  given 
0{u),  I{u)  and  H{u)  are  proportional  to  each  other,  the  conditional  pdf  Pi\o  is  simply 
related  to  the  pdf  Ph  of  the  OTF: 

Pilo(/(«)10(«))  =  Ph{H{u)  =  I{u)IO{u))J^HHII),  (12) 

where  iH{u) /I{u))  is  the  Jacobian  determinant  of  the  transformation  from  the  complex 
H{u)--pl&ne  to  the  complex  7({t)-plane.  Since  0(u)  is  held  fixed  in  the  conditional  pdf 
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calculation,  the  Jacobian  is  simple  and  turns  out  to  have  the  value  l/10(u)p.  The  inverse 
conditional  pdf  Po\i  can  in  turn  be  computed  by  means  of  the  Bayes  theorem, 


Po\i{0{u)\I{u))  = 


p,i„(j(u)|0(tr))Po(0(u)) 

/  Pi\o{I{u)\0{u))Po{0{u))<PO{u)  • 


(13) 


When  other  sources  of  noise  are  present  as  well,  as  for  example  in  photon  counting  at 
low  count  rates,  their  effects  may  once  again  be  integrated  into  a  single  pdf  by  further 
applications  of  the  Bayes  theorem  for  compounded  random  processes. 

When  the  support  constraint  is  applied  in  the  image,  the  image  spectrum  changes 
via  Eq.  (2).  However,  because  of  the  Unearity  of  that  equation,  the  statistics  of  the  post¬ 
support  spectrum  are  rather  simply  related  to  those  of  the  pre-support  spectrum 

I{u).  It  may  be  shwon  under  rather  general  conditions  that  the  statistics  of  the  OTF 
H{u)  are  Gaussian  and  therefore,  by  Eq.  (12),  so  are  the  conditional  statistics  off.  Since  a 
linear  combination  of  Gaussian  random  variates  too  has  Gaussian  statistics,  the  conditional 
statistics  of  the  post-support  spectrum  are  also  Gaussian.  Under  these  conditions,  the 
change  of  mutual  information  AZ(«)  given  by  Eq.  (11)  takes  on  a  more  expUcit  final  form 


0{u) 


- 


j  APi(/(u))iogP((/(a))<?;(S), 


(14) 


where  cr|  denotes  the  variance  of  the  circular  Gaussian  pdf  which  describes  the  fluctuations 
of  the  OTF.  Like  in  Eq.  (11),  the  prefix  A  designates  the  change  in  a  quantity  when  the 
support  constraint  is  applied. 


III.  REPORT  ON  PROGRESS  MADE  DURING  THE  PERIOD  9/98-8/99 


Three  major  accomplishments  were  made  during  this  period.  The  first  concerns  the 
publication  of  the  paper  “Dynamics  of  ttirbulence  induced  noise  in  image  deconvolution 
with  support  constraint”  in  the  J.  of  the  Optical  Society  of  America  (JOSA  A,  vol.  16,  pp. 
1769-1778  (1999)).  In  this  paper,  discussed  in  the  Sec.  II  A  and  accompanying  the  report, 
we  discuss  the  improvement  of  the  information  content  of  the  decovolved  image  in  some 
what  indirect  terms  of  reduction  of  noise  in  the  image  when  a  priori  knowledge  of  the  image 
support  is  used.  The  second  concerns  a  direct  study  of  the  information  transport  under 
the  application  of  support  constraint.  This  paper,  discussed  in  Sec.  II  B  and  attached  here 
in  reprint  form,  was  presented  at  the  Fundamental  Issues  in  Image  Formation,  Detection, 
and  Processing  workshop  organized  by  me  at  UNM’s  Center  for  Advanced  Studies,  Febru¬ 
ary  6-7,  1999.  Numerical  simulations  confirming  the  improvement  of  mutual  information 
are  presented  here  in  detail.  The  third  accomplishment  is  our  explicit  calculation  of  the 
information  capacity  of  an  imaging  system  hmited  by  atmospheric  turbulence  and  additive 
detector  noise.  This  work  [7],  attached  here  in  preprint  form,  was  invited,  presented,  and 
well  received  at  the  recently  concluded  Air  Force  Maui  Optical  Station  (AMOS)  Technical 
Conference,  Aug  30  -  Sep  3,  1999.  A  longer  and  more  complete  paper  [8]  on  this  work 
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was  submitted,  refereed,  and  pubHshed  in  Optics  Communications.  This  work  has  made  it 
possible  for  us  to  begin  to  develop  a  quantitative  imderstanding  of  the  statistical  informa¬ 
tion  content  of  a  priori  knowledge  such  as  support,  which  was  a  key  task  that  we  had  set 
out  to  perform  under  the  present  Contract.  Finally,  a  paper  entitled  “Noise  Transport  and 
Removal  in  a  Speckle  Imaging  Algorithm,”  co-authored  with  David  Tyler, 
at  the  Annual  Optical  Society  of  America  Meeting  in  Santa  Clara,  September  26-30,  1999. 


A,  Preliminary  Simulations  of  Information  Dynamics 


We  have  used  a  variety  of  computer  programs  and  a  telescope  imaging  simulation  code 
to  generate  ensembles  of  data  and  corresponding  mutual  information  plots.  First,  an 
ensemble  of  five  different  scenes  was  synthesized  using  Gaussian  “st  ars”  with  various  widths 
and  relative  intensities.  These  objects  were  used  as  inputs  to  a  simulation  modelling  the 
effects  of  the  turbulent  atmosphere,  telescope  diffraction,  and  finite-resolution  detection. 
The  simulation  generated  ensembles  of  several  hundred  frames  of  noise-corrupted  imagery, 
all  of  which  were  then  processed  using  a  deconvolution  algorithm  (to  increase  resolution) 
and  our  support-constraint  convex-projections  algorithm  (to  remove  noise).  Finely,  the 
mutual  information  over  the  data  and  object  ensembles  was  calculated;  first  for  data 
processed  with  only  the  deconvolution  algorithm  and  then  for  data  processed  with  both 
algorithms.  The  mutual  information  was  calculated  for  each  of  the  central  32x32  pixels  in 
the  image  arrays,  forming  information  “maps”  of  the  image  field.  The  displays  (Fip.  1, 
2  and  3)  presented  here  show  the  spatial  distribution  of  the  statistical  information  in  t  e 
object  set,  mutual  information  at  high  turbulence  {D/ro  =  10),  and  the  change  of  mutual 
information  in  the  image  when  the  support  constraint  is  applied.  The  overall  improvement 
of  MI  by  about  5-10%  is  consistent  with  the  degree  of  equivalent  noise  reduction  under  the 
same  support  apphcation. 


B.  Information  Capacity  of  a  Seeing  Limited  Imaging  System 

In  order  to  gain  fundamental  insights  into  the  nature  of  information  transport  and 
improvement  when  a  priori  constraints  are  applied,  we  must  first  understand  the  maximum 
information  carrying  capacity  of  an  imaging  system. 

We  generalize  our  treatment  of  Sec.  II  to  include  an  additive,  zero-mean,  Gaussian 
random  noise  source, 

I(^)  =0(«)if  («)-!- i?(«),  (1) 

and  to  include  at  once  a  finite  region  of  the  spatial  frequency  plane  in  our  considerations 
The  description  at  the  level  of  individual  pixels  is  not  complete  because  it  ignores  mterpixel 
correlations  over  a  finite  region.  The  definitions  must  be  suitably  modified,  with  functional 
integrals  replacing  ordinary  integrals  on  the  right-hand  sides. 

For  D/ro  »  1,  as  before,  the  OTF  [fl-(u)]  has  Gaussian  statistics,  under  which  the 
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conditional  statistics  of  the  image  spectrum  [/(«)]  conditioned  on  the  knowledge  of  the 
object  spectrum  [0(it)]  also  Gaussian.  This  permits  the  calculation  of  the  conditional 
image  information  entropy  'Hi\o  over  the  complete  system  pass-band,  ju]  <  D/{\f),  as  well 
as  the  image  self  information  Hi  -  and  consequently  the  MI  -  in  terms  of  the  so-far  ^known 
object  spectrum  statistics.  Under  the  constraints  of  normalization  of  probabilities,  fixed 
mean  emissive  power  F  of  objects  over  their  ensemble,  and  fixed  mean  integrated  power 
spectrum  G  over  the  system  pass-band,  we  may  maximize  the  MI  throughput  of  the  system 
by  varying  the  statistical  ensemble  of  objects.  This  maximization  has  been  carried  out  with 
the  help  of  Lagrange  multipliers,  and  leads  to  the  following  expressions  for  the  maximum 
statistical  information  transmitted  by  the  system,  or  its  information  capacity  (details  are 
in  Refs.  7  and  8); 


C  —  Coo  «  -logdet  </,  Qij  =  exp[-{l/2)D(f>{Xf\ui  -  Ujl)], 
when  additive  noise  is  negligible,  and 


C  =  log 


\yfirW^fKAu} 


+  ^log 

i=l 


(  WE\Hi\-^  )  ’ 


(2a) 


(26) 


when  the  additive  noise  dominates  and  turbulence  is  absent.  In  Eq.  (la).  Coo  represents 
the  capacity  under  infinitely  strong  turbulence  for  which  only  the  dc  pixel  carries  any 
information.  The  symbol  P  represents  the  number  of  spatial  frequency  pixels,  each  of  area 
AAu,  within  the  system  pass-band,  Wb/AAu  the  additive  noise  at  each  pixel,  and  Hi  is 
the  value  of  the  (noise-free)  OTF  at  the  ith  pixel.  The  detailed  behavior  of  the  preceding 
expressions  for  capacity  is  discussed  in  Refs.  7  and  8. 


IV.  REPORT  ON  PROGRESS  MADE  IN  THE  FINAL  PERFORMANCE 
PERIOD  9/99-8/00  AND  FUTURE  DIRECTIONS 


Two  key  advances  were  made  dining  the  last  performance  period.  The  first  involved 
completion  of  the  paper  on  the  information  capacity  of  a  seeing  and  additive  noise  limited 
astronomical  imaging  system.  The  paper  was  submitted,  refereed,  and  published  [8]  in 
Optics  Communications  in  April  2000.  Secondly,  we  have  applied  the  notion  of  Fisher 
information,  a  measure  of  information  in  statistical  data  when  estimating  a  particular 
observable,  to  gain  a  solid  understanding  of  the  inverse  relation  between  noise  and  infor¬ 
mation.  Investigating  the  deep  connections  between  the  Shannon  and  Fisher  information 
measures,  as  yet  largely  unexplored  in  the  literature,  still  lies  ahead  of  us,  but  our  compu¬ 
tations  of  the  Fisher  information  under  a  variety  of  imaging  conditions  have  ^ready  given 
us  useful  insight  into  these  connections.  We  now  describe  our,  as  yet  unpublished,  studies 
on  Fisher  information. 
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A-  Fisher  Information  and  Minimum  Estimator  Variance 


Consider  a  set  of  parameters  {^i}  that  we  estimate  by  making  measurements  {xj}  that 
are  typicaUy  statistical  in  character.  Let  the  statistics  of  the  measurements  be  described 
the  probability  density  function  (pdf)  P(xl{0}).  The  Fisher  information  matrix  J  is  defined 
by  specifying  its  ij  element  as 

r  _  /glnP(x|W)^lnP(^IW)\  (1) 

-  y  g.  d9*j  /  ’ 

where  the  triangular  brackets  denote  averaging  over  the  pdf  P  itself. 

The  parameters  Bi  are  inferred  from  the  data  by  means  of  estimators,  suitably  con¬ 
structed  functions  of  the  data,  among  which  unbiased  estimators  occupy  a  special  place. 
For,  their  averages  are  defined  to  be  identical,  by  construction,  to  the  parameters  them¬ 
selves.  If  C  represents  the  covariance  matrix  of  any  unbiased  estimators,  Bi(x)  for  the  ith 
parameter  Bi,  then  C  and  J  obey  the  following  Cramer-Rao  matrix  inequality, 

C>J"^  (2) 

in  the  sense  that  C  -  J-^  is  a  nonnegative  definite  matrix.  A  necessary  condition  of  this 
nonnegativity  is  that  the  variance  of  the  ith  estimator  Bi(x)  ca^ot  be  smaller  than  the 
ith  diagonal  element  of  the  inverse  Fisher  information  matrix  J 

var(0i)  > 

The  inverse  of  the  Fisher  information  thus  provides  a  measure  of  the  minimum  uncertainty 
in  the  determination  of  unbiased  estimators.  It  is  in  this  statistical  estimation  sense  that 
Fisher  information  represents  information.  For  estimators  that  are  biased,  a  similar  resu  t 

as  Eq.  (3)  holds. 

The  imaging  problem  can  be  regarded  as  an  estimation  problem  in  which  statistical 
image  data  are  employed  to  infer  the  object  being  viewed.  This  approach  has  been  em¬ 
ployed  in  the  last  few  years  to  analyze  certain  aspects  of  image  formation,  detection,  and 
processing,  but  its  relation  to  Shannon  information  is  not  well  understood.  It  is  this  rela¬ 
tion  which  we  must  elucidate  before  we  can  hope  to  clarify  the  connection  between  noise 
reduction  and  information  gain.  We  have  recently  turned  our  attention  to  this  problem. 

We  describe  below  our  results  for  the  class  of  estimators  in  which  image  data  at  a 
single  spatial  frequency  are  used  to  estimate  the  object  spectrum  a,t  that  frequency.  The 
Fisher  information  matrix  reduces,  at  each  spatial  frequency,  to  a  single  number  that  can 
be  easily  computed  under  certain  approximations. 

For  D/ro  >>  1  and  in  the  mid-frequency  range,  the  OTF  has  nearly  Gaussian  statis¬ 
tics,  and  so  are  the  conditional  image  statistics,  given  an  object,  statistics  that  determine 
Fisher  information: 

P{Ij\Oj)  =  -^exp  [-{Ij  -  Oj{Hj)){Ij  -  Oj{Hj))ICjj]  ,  (4) 
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where  {Hj)  is  the  mean  OTF  at  spatial  frequency  Uj  and  the  image  amplitude  spectrum 
variance  at  that  frequency  is 

Cjj  =  \Oj\^cTl{uj)  +  a%.  (5) 

The  first  term  on  the  right  hand  side  of  Eq.  (5)  represents  the  fiuctuation  in  image  data 
due  to  atmospheric  turbulence  and  other  OTF  fluctuations,  their  variance  having  been 
denoted  by  while  the  second  term  represents  the  additive  zero-mean  detector  noise 
taken  to  be  independent  of  spatial  frequency. 

For  the  Gaussian  pdf  (4),  the  computation  of  Fisher  information  is  straightforward. 
For  the  case  that  the  object  spectral  amplitude  Oj  is  purely  real,  we  have  for  the  reciprocal 
of  Fisher  information 

,  /rainP(jj|Oj)1^\ 

~\[  dOj  J  / 

O]  \1  +  1/{SNR-<tI)]^  (6) 

~  4  [l-\-l/{SNR-cTiyY 

where  SNR  is  the  power-spectrum  “signal  to  noise  ratio”  at  the  spatial  frequency  Uj, 

Oj  (7\ 

SNR  =  (7) 

The  OTF  variance  cr\  tmder  the  conditions  we  have  assumed,  D/ro  »  1  m  the  mid- 
frequency  range,  is  nearly  the  same  as  the  speckle  tranfer  function,  and  takes  the  value 

a\  «  0.435  *  {tqIDY  * 

where  Ho{uj)  denotes  the  OTF  of  the  telescope  in  the  absence  of  turbulence,  i.e.,  the 
diffraction-limited  OTF,  at  frequency  uj. 

In  Fig.  4,  we  plot  the  result  (6),  which  represents  the  best  lower  bound  on  the  variance 
of  an  unbiased  estimator  of  the  object  spectrum  at  that  frequency.  It  is  clear  that  for  each 
D/ro  value,  as  SNR  increases,  the  minimum  estimator  variance  decreases  as  expected. 
For  the  same  SNR,  on  the  other  hand,  the  minimum  estimator  variance  increases  with 
Dfro,  once  again  as  expected. 

Against  the  best  theoretical  performance  (6)  predicted  by  the  Fisher  information 
analysis,  we  may  compare  the  actual  variances  of  certain  well  known  estimators.  We 
considered  three  estimators,  namely  the  inverse  filter,  the  Wiener  filter,  and  the  filtered 
back-projection  estimator.  Only  the  first  of  the  three  is  an  unbiased  estimator,  while 
the  third  is  so  only  when  SNR  is  infinitely  large  {(tn  =  0),  but  a  simple  multiplicative 
correction  to  Fisher  information  is  all  that  is  needed  to  include  a  nonzero  bias  when 
computing  the  theoretical  minimum  bound  on  estimator  variance. 
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In  the  next  figiure,  Fig.  5,  we  present  a  plot  of  the  minimum  estimator  variance  for  the 
case  of  atmospheric  turbulence  alone,  as  a  function  of  spatial  frequency.  With  increasing 
Dfro,  the  minimum  variance  rapidly  approaches  the  value  \0j\^/ 4,  given  by  Eq.  (6)  under 
zero  additive  noise  {SNR  =  oo). 


B.  Future  Studies 


There  are  several  directions  in  which  Fisher-information-based  studies  of  the  perfor¬ 
mance  of  an  imaging  system  are  being  conducted  by  us  at  present.  Among  them,  the  three 
most  notable  directions  are  the  following: 

1.  The  connection  between  Fisher  and  Shannon  information  must  be  fully  understood 
before  noise-based  and  information  based  limits  of  performance  can  be  meaningfully 
compared.  There  is  a  slight  conceptual  difficulty  in  finding  this  connection  -  Fisher 
information  governs  the  precision  of  statistical  measurements  of  one  or  more  fixed 
parameters,  while  Shannon  information  applies  when  statistical  distributions  of  para¬ 
meters  are  involved.  By  extending  the  defimtion  of  Fisher  information  to  statistically 
specified  parameters,  the  two  measures  of  information  can  both  be  made  to  apply  to 
the  imaging  problem.  This  is  the  approach  we  a.dopt.  Our  early  results  are  encourag¬ 
ing,  but  thorough  simulations  are  needed  for  complete  understanding. 

2.  The  dynamics  of  Fisher  information  under  application  of  prior  knowledge,  e.g.,  sup¬ 
port,  are  well  worth  investigating.  Such  studies  will  enable  us  to  understand  how 
prior  knowledge  of  a  certain  kind  affects  the  distribution  of  information/noise  in  the 
physical  and  spatial-frequency  planes.  Like  our  Shannon  information  based  work, 
simulations  are  needed  to  understand  these  dynamics  well. 

3>  Iterative  image  processing  techniques  like  Lucy  Richardson  and  expectation  maxi¬ 
mization  (EM),  and  maximum  a  posteriori  probability  (MAP)  methods  aU  suffer  from 
the  quiet  indignity  of  having  no  robust  stopping  criteria.  These  techmques,  if  left 
to  converge,  lead  to  meaningless  data-independent  reconstructions.  Both  Shannon 
and  Fisher  measures  of  information  have  the  promise  of  furmshing  objective  stopping 
criteria  to  decide  at  what  iteration  number  to  regard  the  reconstruction  as  having 
yielded  the  ’’best”  image.  The  best  image  must  maximize  information  contained  in 
the  measured  data  and  prior  knowledge  of  the  object  attributes  (Uke  support,  posi¬ 
tivity,  etc). 
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